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The effect of student evaluations of teaching on 
administrative personnel decisions at the Department of Psychology, 
University of Western Ontario, was investigated. In addition, the 
apparent impact of summative student evaluation of teaching upon 
faculty attitudes and faculty teaching performance was assessed. In 
the Department of Psychology, undergraduate student evaluations are 
obtained by a 10-item questionnaire that focuses mainly on 
expositional skills such as clarity, preparation, and use of 
examples, and includes a final overall effectiveness item. It was 
found that: student evaluations of teaching contributed significantly 
to decisions on faculty salary and tenure, but not to decisions on 
promotion to full professor; after 12 years of mandatory summative 
evaluation, faculty members continue to have generally favorable 
attitudes toward student evaluations of teaching and their use in 
administrative personnel deci„sions; and the mean level of teaching 
effectiveness of the Department of Psychology as a whole has improved 
substantially over the past 12 years, presumably due, at least in 
part, to the incentive and selection functions of summative 
evaluation of teaching. It is suggested that a fully effective 
teaching evaluation system must include both formative and summative 
components in order to effect the right combination of feedback, 
motivation, training, and selection needed to gain faculty acceptance 
and to produce significant improvement in individual and 
institutional performance. (SW) 



ED 223 162 

AUTHOR 
TITLE 

PUB DATE 
MOTE 

PUB TYPE 

EDRS PRICE 
DESCRIPTORS 

IDENTIFIERS 
ABSTRACT 



*********************************************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
*********************************************************************** 




Use of student instructional ratings in administrative 



personnel decisions at the University of 



Western Ontario 



1 



• PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



U.S, OEMRTMEPIT OF EDUCATION 

NATIONAL INSTITUTE OF EDUCATION 



VrT^fh^ document has been rtproduced as 
receivtd from the person or organization 

onginating it. 



EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 



Harry G. Murray 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC)." 



University of Western Ontario 



n M.nor changes have been mada to Improve 
reproduction quality- 



• Points of view or opinions statad in this docu- 



ment do not nacessarilv raprasent official NIE 



position or policy. 



Introduction 

Recent surveys (e.g.. Centra, 1977) indicate that nearly all North 
American universities now have some sort of plan for systematic evaluation of 
teachers and teaching, with formal student ratings representing the most 
common method of evaluation in current use. In most universities, teaching is 
evaluated for one or more of the following purposes: (1) to provide faculty 
members with diagnostic feedback which may lead to improved teaching 
performance - usually termed formative evaluation ; (2) to provide data to be 
used in administrative decisions on faculty hiring, salary, contract renewal, 
tenure f and promotion - generally referred to as summative evaluation ; and 
(3) to provide published information to be used by students in selecting 
optional courses. 

Although there is little or no controversy about evaluation of teaching 
for purely formative purposes, or even for student use in course selection 
provided that faculty members participate voluntarily, the use of teaching 
evaluation data, and student ratings in particular, as a determinant of 
salary, tenure, and promotion decisions continues to be a matter of^lieated 
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controversy, both in the professional literature and in the faculty coffee 
lounge. One aspect of this controversy is the question of whether teaching 
evaluations do in fact make a difference in personnel decisions. Some 
observers claim that, despite administrators' public pronouncements on the 
importance of teaching, teaching evaluations are all but ignored in salary, 
tenure, and promotion decisions. Other observers argue (usually in private) 
that considering the extreme fallibility of existing teaching evaluation 
procedures, teaching is already given far too much weight in the faculty 
advancement system! A second aspect of the controversy relating to summative 
evaluation of teaching has to do with the actual or potential effects of such 
a policy upon faculty morale, faculty-student interaction, and teaching 
performance itself. Advocates of summative evaluation (e.g., Hildebrand, 
1972) argue that incorporation of teaching evaluation data into the faculty 
advancement system motivates faculty members to take teaching seriously and 
devote the time and effort needed to bring about improvement in performance. 
Opponents of summative evaluation (e.g., Scheck, 1978) contend that this 
practice causes anxiety, poor morale, and divisiveness among faculty members, 
and therefore is counterproductive as a means of improving performance. 
Furthermore, and perhaps of even greater concern, it is claimed that the use 
of student instructional ratings in administrative personnel decisions induces 
faculty members to raise grades and lower academic standards in an attempt to 
"buy" favorable evaluations from students. 

The purpose of the present research was to investigate whether or not 
student evaluations of teaching have contributed significantly to 
administrative personnel decisions in the Department of Psychology, University 
of Western Ontario; and furthermore, to assess the apparent impact of 
summative student evaluation of teaching upon faculty attitudes and faculty 
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teaching performance. The University of Western Ontario was one of the first 
institutions in North America to introduce a mandatory campus-wide program of 
student evaluation of teaching for administrative purposes. Since 1970^ 
Senate policy has required that evaluations of teaching be solicited from 
undergraduate and graduate students and from faculty colleagues , and that 
these evaluations be considered in decisions on faculty retention, tenure^ and 
promotion. The specific procedures for soliciting student and colleague 
evaluations of teaching are left to the discretion of individual faculties and 
departments, as is the relative weighting of teaching, research, and service 
evaluations in personnel decisions. Contrary to the intention of Senate 
policy, most departments have failed to develop effective procedures for 
obtaining evaluations from graduate students and colleagues, with the result 
that nearly all documentation on teaching at UWO comes from undergraduate 
students. In the Department of Psychology, undergraduate student evaluations 
are obtained by means of a 10-item questionnaire which focuses mainly on 
expositional skills such as clarity, preparation, and use of examples, and 
includes a" final "overall effectiveness" item. The questionnaire is 
administered annually in all undergraduate courses, with students responding 
anonymously, the instructor absent during the evaluation period, and 
evaluation results released only after final grades have been submitted to the 
Registrar. Copies of the evaluation results are sent to the instructor, the 
department chainaan, the Dean, and, provided the instructor consents, to the 
University Students' Council for publication in an annual teacher evaluation 
booklet. Teaching evaluation results sent to the department chairman and Dean 
automatically become part of the documentation considered by department- and 
faculty-level promotion and tenure committees • The research reported here 
addressed three specific questions concerning the use of student evaluations 
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of teaching in administrative personnel decisions in the Department of 
Psychology: 

1. Do student evaluations make a difference in personnel decisions? 

2. Are faculty members satisfied or dissatisfied with the use of 
student evaluations in personnel decisions? 

3. Does the use of student evaluations in personnel decisions lead 
to improvement in faculty teaching performance? 

Impact of student evaluations upon personnel decisions 

The question of whether or not student evaluations of teaching have 
contributed significantly to administrative personnel decisions was 
investigated separately for three different types of personnel decisions: 
(a) awarding of annual merit salary increments, (b) granting of tenure, and 
(c) promotion to full professor. In the case of salary adjustments, the 
answer to this question is obviously affirmative, in that the policy of the 
Department ^of_Psychology is to weight teaching, research, and service 
evaluations 40,40 and 20% respectively in the determination of the merit 
component of annual salary increments. Thus, depending on the apportionment 
of merit and across-the-board components of the overall salary settlement, 
good vs. poor student evaluations could make a difference of perhaps $200 to 
$400 in a faculty member's salary increment, a difference which is, of course, 
cumulative over subsequent year^ of employment. 

Evidence that student evaluations of teaching have contributed 
significantly to tenure decisions in the Department of Psychology is presented 
in Figure 1, which shows a scatterplot of mean student instructional ratings 
(5-point scale of "overall effectiveness"; averaged across courses and years) 
and mean publication rates (weighted scale similar to that of Hoyt, 1974; 
averaged across years) for 28 faculty members appointed to tenure-track-^ 



positions between 1970 and 1976 who had either been granted tenure (N=16) or 
denied tenure (N=12) as of July, 1979. Faculty members who resigned after 5 
or more years service without gaining tenure were classified as "denied 
tenure" for purposes of this analysis; whereas faculty members who resigned 
without- tenure after 4 years or less, and faculty members for whom tenure was 
pending as- of July , 1979 were omitted from the analysis.- The mean teaching 
and research scores shown in Figure 1 were computed from the time of 
appointment to the time of a positive or negative tenure decision (or 
resignation). It may be noted in the figure, first, that teaching and 
research scores were statistically uncorrelated (r = .07) and therefore served 
as independent sources of information in personnel decisions; second, that 
student instructional ratings did indeed make a difference in tenure 
decisions, in the sense that the mean rating of faculty members granted tenure 
was significantly higher (t - 2.00, p < .05) than the mean rating of those 
denied tenure; and third, that student ratings contributed considerably less 
to tenure decisions than did rate of publication. In support of these 
conclusions, a stepwise multiple regression analysis showed that student 
instructional ratings accounted for approximately 12% of variance i rate of 
publication approximately 60%, and factors such as age, sex, and nationality 
a total of less than 3% of the variance in tenure decisions in the Department 
of Psychology between 1970 and 1979. 

An analysis similar to the above showed that student instructional 
ratings did not contribute significantly (less than 2% of variance accounted 
for) tp, administrative decisions on promotion to full professor. 
Faculty attitudes 

Given that student evaluation of teaching do seem to make a difference, 
at least in salary and tenure decisions, one might ask whether faculty members 



are happy or unhappy with this state of affairs. Although data on faculty 
attitudes are not available for the Department of Psychology per se , data are 
available for the Faculty of Social Science, which includes the Department of 
Psychology, and which uses teaching evaluation procedures similar to those of 
Psychology. Table 1 summarizes the responses of 126 Social Science faculty 
members (a return rate of approximately 58%) to selected attitudinal 
statements on student instructional ratings and their use in administrative 
personnel decisions. Faculty attitudes on these matters are perhaps best 
described as "mixed, but generally positive". As may be noted in Table 1, a 
clear majority (69%) of faculty respondents continued to support the idea of 
compulsory campus-wide evaluation of teaching for summative purposes, and a 
similar majority (61%) stated that student evaluations of teaching are 
acceptable for use' in administrative personnel decisions. Furthermore, only 
33% of faculty respondents felt that the use of student instructional ratings 
in personnel decisions causes "grade inflation", and only 12% felt that this 
practice leads to deterioration of faculty-student interaction. On the other 
hand, less than half (40%) of faculty members were "basically satisfied" with 
the 10-item student rating form used in the Faculty of Social Science; less 
than half (42%) believed that the student rating form provides diagnostic 
feedback useful for improvement of teaching, and less than half (44%) , ire in 
favor of placing increased weight on teaching in salary, tenure, and promotion 
decisions. On a more positive note, 73% of respondents stated that the Social 
Science student rating form provides valid information on certain aspects of 
teaching, and 93% agreed that it is within the realm of possibility to develop 
a "valid" student rating form. All in all, one gets the impression that 
although faculty members are not entirely satisfied with existing student 
evaluation procedures, they continue to be generally favorable to the idea of 
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using student instructional ratings for administrative purposes, and have not 
given up on the possibility of developing a more satisfactory student 
evaluation system. 

Improvement in teaching performance 

Although improvement of performance is supposedly not the primary goal of 
summative evaluation of teaching, there are at least three reasons for 
expecting that incorporation of student evaluations into the faculty 
advancement system should lead to significant improvement in teaching 
performance: (1) student ratings provide feedback, which may be useful in 
diagnosing problems and developing more effective teaching strategies; (2) use 
of student evaluations for summative purposes provides faculty members v/ith a 
tangible incentive for putting time and effort into improvement of teaching; 
and (3) use of student evaluations in tenure decisions provides a selection 
procedure whereby excellent teachers care more likely to be granted tenure and 
retained by the department (see Figure 1). As may be noted in Figure 2, there 

is clear evidence that the mean level of teaching effectiveness of the 

.... ■ ^ 

Department of Psychology as a whole (as indexed by student ratings) has in 
fact improved significantly over the years from 1970 to 1982. Unfortunately 
it is next to impossible with the data at hand to determine whether this 
imprpvement in teaching performance resulted from the feedback r incentive, or 
selection functions of summative evaluation, or perhaps from some combination 
of these factors. It is equally plausible that the results shown in Figure 2 
are attributable, a least in part, to some totally extraneous factor, such as 
an increase over successive years in the "leniency bias" of subject 
instructional ratings. An interesting supplementary finding is that the mean 
teaching performance of faculty members who already had tenure in 1970 (N=12) 
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showed no corresponding tendency to improve over the period from 1970 to 1982. 
Another interesting finding, depicted in Figure 3, is that faculty members 
granted tenure between 1972 and 1977 (N=13) tended to improve steadily in 
teaching performance from the year of appointment to the year in which tenure 
was achieved f but thereafter showed a noticeable decline in student ratings^ 
followed by a levelling-of f and partial recovery. The latter results suggest 
that teaching experience and student feedback alone cannot account for the 
improvement in departmental mean teaching performance seen in Figure 2. 
Whereas Cohen (1980) concluded that feedback from student ratings is not 
particularly effective in improving faculty teaching performance unless 
accompanied by expert coaching or consultation, the present findings suggest 
that student feedback is not particularly effective in improving teaching 
performance unless evaluation results are effectively incorporated into the 
faculty review and advancement system. 
Conclusions 

The results of this study indicate: (1) that student evaluations of 
teaching contribute significantly to decisions on faculty salary and tenure, 
but not to decisions on promotion to full professor, in the Department of 
Psychology, University of Western Ontario; (2) that after 12 years of 
mandatory summative evaluation, faculty members continue to have generally 
favorable attitudes toward student evaluations of teaching and their use in 
administrative personnel decisions; and (3) that the mean level of teaching 
effectiveness of the Department of^ Psychology as a whole has improved 
substantially over the past 12 years, presumably due at least in part to the 
incentive and selection functions of summative evaluation of teaching. 
Despite the common tendency to interpret formative and summative evaluation of 
teaching as mutually exclusive alternatives, the present research suggests 
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that a fully effective teaching evaluation system must include both formative 
and summative components. Formative evaluation used in isolation is unlikely 
to produce significant improvement in teaching because tangible incentives for 
improvement are lacking, as is the opportunity to select and retain excellent 
teachers. Summative evaluation used in isolation is likely to produce ^ 
resentment in faculty members who receive poor teabhing evaluations but have 
no avenues for improvement. Only when summative and formative evaluation are 
used in combination do we have the right combination of feedback, motivation, 
training, and selection needed to gain faculty acceptance and to produce 
significant improvement in individual and institutional performance. 
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Table 1 

Faculty attitudes toward student instructional 
ratings and their use in administrative 
personnel decisions 



Percent responding 



Statement Un- Dis- 

Agree decided agree 



The use of teaching evaluations in 
salary, promotion, and tenure 
decisions should be compulsory for 
all faculty members. 

The use of teaching evaluations 
in salary, promotion, and tenure 
decisions motivates faculty members 
to put time and effort into 
improvement of teaching. 

Student evaluations of teaching 
are acceptable for use in salary, 
promotion, and tenure decisions. 

I am basically satisfied with the 
questionnaire used for student 
evaluation of teaching in my 
department. 

The student rating form used in 
my department provides valid 
information on certain aspects 
of teaching. 



69 



00 



31 



54 



19 



27 



61 



40 



00 



12 



39 



48 



73 



11 



16 



The student rating form used in 
my department provides useful 
feedback for improvement of 
teaching. 

The use of student ratings af 
teaching in salary, promotion, and 
tenure decisions leads to grade 
inflation and lowering of academic 
standards. 



42 



23 



35 



33 



24 



43 
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Table 1 
continued 



Percent responding 


Statement 


Agree 


Un- Dis- 
decided agree 


The use of student ratings of 
teaching in salary^ promotion, and 
tenure decisions leads to 
deterioration of faculty - 
student interaction. 


12 


^"22 66 


It is within the realm of possibility 
to develop a "valid" rating form for 
student evaluation of teaching. 


93 


00 07 


The teaching evaluation system should 
be broadened to include sources of 
data other than undergraduate 
student ratings. 


65 


16 19 


More weight should be placed on teaching 


44 


12 44 



evaluations in salary, promotion, and 
tenure decisions. 



0 GRANTED TENURE 
O DENIED TENURE 



ul 
< 



20.00 



10.00 — 



z 

o 

< 

Z 6.00 

CO 

a. 



1 



5.00 



If. 0 0 — 



+ 



O 



—I 

3.00 3.20 3.H-0 
TEACHER RATING--^ 

O 



4- 



3.60 
3.00 



2.00 — 



1.00 ~ 



O 



O 



0.00-- 



o 



3.80 



-4— 

4-.00 

o 



f.20 



•hH-O 



O 
O 



ERIC 



Figure 1. Scatterplot of teacher ratings and publication rates 
of faculty members granted tenure (N=16) and 
denied tenure (N=12) between 1970 and 1979. 
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Figure 2. Department mean teacher rating score for 
' academic years 1969-70 to 1981-82. 



16 



TENURE 
G-RANTED 



>.10 - 



I- 

DC 



ai 
X 

< 
I- 

z 

< 




»f-.oo - 



3.90 - 



3.80 - 




■ 17 



18 



Figure 3. Mean teacher ratings in three pre-tenure 
and four post-tenure years for 13 faculty 
members granted tenure between 1972 and 1977. 



